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Abstract 

This paper describes an approach to 
identify factors that significantly 
influence microgravity experiment 
performance . Investigators developed 
the "Degrees of Success" scale to 
provide a numerical representation of 
success . A degree of success was 
assigned to 293 microgravity 
experiments . * Experiment information 
including the degree of success rankings 
and factors for analysis was compiled 
into a database. Through an analysis of 
variance, nine significant factors in 
microgravity experiment performance were 
identified. The frequencies of these 
factors are presented along with the 
average degree of success at each level. 
A preliminary discussion of the 
relationship between the significant 
factors and the degree of success is 
presented . 

Introduction 

Shuttle-based microgravity experiment 
performance is an important 
consideration given the time, effort, 
and cost involved in space research. 
The reliability of these experiments is 
currently determined at the micro 
(component) level. The reliability of 
these components are determined either 
by testing, historical performance data 
or by vendor data, which may include 
both testing data and performance data. 
These reliability measures are then used 
to calculate a system reliability using 
well documented mathematical probability 
models. The risk to mission success is 


then based on the value of the system 
reliability. This process overlooks the 
external effects on the system 
reliability with exception to component 
derating. This study is an 
investigation into those external 
effects; an approach to the system 
(experiment) reliability from a macro- 
level. 1 

Several studies have addressed 
microgravity experiment performance. In 
a 1986 study of 117 active Get Away 
Special (GAS) experiments, Rex Ridenoure 
found that 40 percent of these 
experiments obtained all of the desired 
data. 2 In another study, Cheryl Winter 
and Jonathan Jones found that 64 percent 
of the 100 experiments examined from 
their fluids and materials processing 
database obtained at least 75 percent of 
the desired data. 3 Having both 
presented a significant number of 
experiments with anomalies, these 
studies show that an effort to enhance 
the performance of microgravity 
experiments is worthwhile. 

These two studies have identified types 
of experiment failures and quantified 
the percentage of experiment objectives 
affected. However, there is a need to 
identify the factors that significantly 
influence experiment performance, as 
discussed by Thaggard and Morilak. 4 To 
satisfy this need requires three main 
actions: (1) define successful 
performance, (2) select factors for 
analysis, and (3) compile information 
for analysis. Investigators for this 
project have created a numerical 
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representation of successful experiment 
performance, the "Degrees of Success” 
scale and selected 26 factors from a 
list compiled by Thaggard and Morilak in 
1991. Also, investigators have compiled 
a database of 391 experiments, the Lewis 
Research Center Microgravity Database 
(LMDB) . Since these actions have been 
taken, it is possible to determine the 
significance of factors that influence 
experiment success. Principal 

investigators (Pis) may use the results 
of the data analysis presented in this 
report during preflight development to 
enhance experiment performance. 

In addition to the results of data 
analysis, this report describes the 
Degrees of Success scale and the 
methodology used to develop the LMDB. 
In preview of the LMDB description, this 
project excludes the following four 
categories of experiments: (1) 

experiments flown prior to 1981 or after 
1991, (2) GAS experiments, (3) 

experiments flown on the Long Duration 
Exposure Facility (LDEF) , and (4) 
student experiments .. Because experiment 
results were frequently unavailable, 
only 293 of the 391 LMDB experiments are 
included in the present evaluation. 
Criteria for exclusion are explained in 
a later section of this paper. 

Methodology 

Select Performance Factors 

As the first major requirement of this 
research, Thaggard and Morilak compiled 
a list of 32 factors that could 
influence microgravity experiment 
performance . Table I shows an 

alphabetical listing of the 26 factors 
that are included in this analysis. 
Factor definitions are in Appendix B. 
Modifications to the original list of 
factors were made through the course of 
this study as investigators found that 
data was unavailable for these factors, 
and that some factors were redundant. 

Modifications to the list of factors 
include the addition of one category, 
type of orbital crew involvement, and 
the elimination of six categories, set- 
up time, upweight, downweight, 
experiment duration, hardware, and 
apparatus. Also, two categories were 
combined, storage container and storage 
location into a new category, storage 
location. This new category is a more 


detailed description of storage location 
than the original definition. Also, 
changes to the factor levels were made. 

Table I Performance Influencing Factors 


Experiment Specific Factors: 

Active (Yes/No) 

Altitude Requested 
Experiment Location 
Experiment Type 
Failure Detection 
Inclination Requested 
Interface 
Iteration 

Level of Ground Crew Involvement 

Level of Orbital Crew Involvement 

Minimum Time On-orbit 

Number of Lockers 

Organization 

Storage Location 

Type of Orbital Crew Involvement 

Mission Specific Factors : 

Crew Size 

Flight Altitude 

Flight Duration 

Flight Inclination 

Launch Delay Cause 

Launch Delay Duration 

Number of Experiments On-board 

Orbiter 

Orbiter Pad Weather 

Time Between Previous Flight 

Wait Time on Pad 


As previously stated, the main reason 
that some factors have been eliminated 
is because the required information 
could not be found in the available 
documents. For instance, data for set- 
up time, the time taken to unstore the 
experiment and initiate operation, was 
not available for any of the LMDB 
experiments . 

Two other factors that were eliminated 
are upweight and downweight. These 
factors represent experiment weight 
during launch and landing respectively. 
These categories were eliminated for 
three reasons: (1) weights could not be 
found for nearly 60 percent of the 
experiments, (2) a strong correlation 
was found between experiment weight and 
experiment location, thus experiment 
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Missing Data Status 


weight duplicates information specified 
in experiment location, and (3) no 
variation was found between upweight and 
downweight . 

The remaining factors, experiment 
duration, hardware, and apparatus, were 
eliminated for the following reasons: 

(1) nearly 80 percent of the experiments 
were missing experiment duration data, 

(2) hardware and apparatus did not 
provide any useful information for this 
study as defined in the original list of 
factors. None of the six factors that 
have been eliminated could be used in 
analysis . 

Compile Database 

Establish criteria 

After the list of factors was developed, 
the next step was to compile the LMDB . 
Because of the massive amounts of 
information that has been published on 
microgravity experiments to date, it was 
necessary to establish guidelines. As 
previously mentioned, four categories of 
experiments were not included in the 
database. The reasons these categories 
were not included are as follows: (1) 
based on the first Space Shuttle mission 
being in 1981 and a chosen 1991 cutoff 
date based on availability of 
microgravity experiment data, the 1981- 
1991 time window was established, (2) 
GAS experiments were thoroughly reviewed 
by Rex Ridenoure in 1986, and (3) LDEF 
and student experiments were not 
included because they are too unique for 
their analysis to have broad-based 
applications . 4 

v LMDB Development 

Having established criteria for LMDB 
experiments, the LMDB was developed and 
includes 52 fields and 391 experiments. 
The main sources for experiment 
information were NASA RECON, ARINC, and 
the AMPTS database maintained at NASA 
JSC. By the end of 1992, there were 
approximately 370 experiments in the 
database. 4 Since then, 30 experiments 
have been added. The source of most of 
these additional experiments was the 
MS AMS database compiled by Winters and 
Jones at NASA MSFC which is now 
available through a computational 
network . 3 


Most records in the LMDB are complete in 
general experiment information such as 
experiment objectives. Similarly, 
mission specific information such as 
flight altitude, flight inclination, and 
crew size are readily available. 
However, experiment specific information 
such as weight, duration, and the 
prelaunch specifications, minimum time 
on-orbit, requested altitude, and 
requested inclination, has proven 
difficult to obtain. Therefore, there 
is a high percentage of missing data in 
these experiment specific areas. In 
addition to missing a high percentage of 
some experiment specific information, 
results were missing for around 100 
experiments. Most of the Pis contacted 
to find this information indicated that 
these results had not been published. 
Based upon the ability to obtain the 
necessary results to assign a degree of 
success, the present analysis includes 
293 experiments. 

The Degrees of Success Scale 
Background 

The last requirement in this research 
was to define successful performance. 
Rex Ridenoure defined success of an 
experiment as achieving all of the 
objectives, and failure as achieving 
none or only part of the objectives. 2 
In the study conducted by Cheryl Winter 
and Jonathan Jones, success was defined 
in four basic ways: (1) just being able 
to fly an experiment aboard the Space 
Shuttle, (2) no anomalies, (3) meeting 
the objectives, and (4) advancing the 
materials science field. 3 The measure 
of successful performance in this 
project, the Degrees of Success scale, 
incorporates many aspects from these 
earlier studies; however, it is 
numerical, rather than categorical. 

Development 

The Degrees of Success scale is based on 
the following criteria: (1) objectives, 
(2) results, (3) problems encountered 
and (4) the minimum success requirement, 
achieving at least one objective (Figure 
1) . Based on these criteria, an 
experiment that achieved the full 
objective without any problems would 
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Figure 1 Degrees of Success Scale 
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encountered . 
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Additional 
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receive a "9”. Similarly, an experiment 
that achieved the full objective without 
any problems, and also obtained data 
beyond the full objective would receive 
a ”10”. On the other hand, an 
experiment that was not attempted at all 
would receive a "l" or ,, 2 ,t depending on 
whether problems were related to design. 
An experiment would receive “1" for a 
design problem. For mid-range values of 
the Degrees of Success scale, the type 
of problem encountered is also taken 
into account where experiments 
encountering a problem unrelated to 
design or development receive a higher 
rating. Despite the subjective nature 
of the degree of success scale, this 
numerical definition of success offers 
opportunities for statistical analysis 
which non -continuous , categorical 
definitions cannot provide. 

Data Analysis 

As previously mentioned, 293 of the 391 
LMDB experiments were assigned a degree 
of success and used in the data 
analysis. For each of the 26 factors 
that were selected, the frequency of 
their levels in the 293 experiments was 
identified. Also, an Analysis of 
Variance (ANOVA) was conducted to 
determine the significance of the 
factors that influence experiment 
success using SPSS PC+, version 5.0. 

The frequency of each degree of success 
ranking is shown in Table II. 
Approximately 44 percent of the 
experiments are ranked 9, and 22 percent 
are ranked 8 . Including the experiments 
ranked 10, around 70 percent of the 
experiments achieved their full 
objectives. On the other hand, around 8 
percent of the experiments did not meet 
their minimum success requirements. 


Table II Frequency of Degree of Success 
Rankings 


Rank 

Frequency 

Percent 

1 

1 

0.3 

2 

1 

0.3 

3 

5 

1.7 

4 

16 

5.5 

5 

24 

8.2 

6 

18 

6.1 

7 

27 

9.2 

8 

63 

21.5 

9 

128 

43.7 

10 

10 

3.4 


The ANOVA produced the following 
results. Four factors, failure 
detection, active, altitude requested 
and experiment type, are significant at 
the 0.01 level (99 percent confidence 
level) . Moreover, level of ground crew 
involvement, minimum time on-orbit and 
storage location are significant at the 
0.05 level, and level of orbital crew 
involvement and orbiter pad weather are 
significant at the 0.1 level. The 
results of the ANOVA for each factor are 
shown in Appendix A. 
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Discussion 

For the nine factors which significantly 
influence experiment performance, the 
frequencies of their levels are 
identified along with the average degree 
of success at each level . This data is 
presented in Tables III-XI. 

Table III shows the frequency counts and 
average degree of success for each level 
of the failure detection. The main 
purpose of the failure detection factor 
is to determine whether detecting a 
problem on-orbit rather than post- flight 
will influence the performance of an 
experiment . The more problems 
encountered by an experiment, the lower 
the average degree of success. In 
addition to the influence of the 
frequency of problems , there was a 
slight difference between the degree of 
success ranking of experiments with 
problems detected on-orbit and those 
with problems detected post -flight. 

As shown in Table IV, of the 293 
experiments assigned a degree of 
success, 34 were passive (required no 


crew involvement) and 259 were active 
(automated or required crew 
involvement) . Passive experiments 
received a higher average degree of 
success than the active experiments. 

The frequency of the different types of 
LMDB experiments along with the average 
degree of success for that type of 
experiment are given in Table VI. The 
experiments with the highest average 
degree of success are Environments and 
Biological experiments while the 
experiments with the lowest average 
degree of success are Metals/Alloys and 
Crystal Growth. 

Table X shows the frequency and average 
degree of success of the storage 
location levels. Experiments stored on 
the spacelab pallet and self-contained 
experiments had a high average degree of 
success . 

Table XI shows the frequency of the 
levels of launch pad weather and their 
average degree of success. The results 
indicate that the average degree of 
success increases along with increases 
in temperature. 
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Table III Frequency and Average Degree of Success 
for Failure Detection 


Level 

IBB9H 

Average Degree of Success 

Problem Detected On-orbit 

66 

6.5 

Problem Detected Post -flight 

62 

6.6 

Problems Detected On-orbit and 
Post-flight 

8 

5.6 

No Problems Reported 

138 

9.0 

Missing 

19 

6.4 


Table IV Frequency and Average Degree of Success 

for Active 


Level 

Frequency 

Average Degree of 
Success 

Passive 

34 

8.5 

Active 

259 

7.6 


Table V Frequency and Average Degree of Success 
for Altitude Requested 


Level 

Frequency 

Average Degree of 
Success 

130 km 

5 

8.4 

135 

47 

7.2 

137 

6 

5.8 

150 

3 

7.3 

160 

13 

8.7 

175 

67 

7.7 

186 

15 

8.2 ' 

190 

27 

8.3 

200 

1 

9.0 

250 

4 

9.3 

Missing 

105 

7.6 
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Table VI Frequency and Average Degree of Success 
for Experiment Type 


Level 

Frequency 

Average Degree of 
Success 

Hardware / Ins t rumen t s 

16 

7.3 

Metals /Alloys 

44 

6.9 

Biological 

69 

8.2 

Fluids & Chemicals 

37 

7.6 

Environments 

13 

8.5 

Crystal 

Growth / Cry s tall ogr aphy 

33 

7.2 

Astronomy 

17 * 

7.9 

Photography 

11 

7.9 

Radiation 

13 

8.0 

Orbiter 

40 

7.7 



Table VIII Frequency and Average Degree of Success 
for Level of Ground Crew Involvement 














































Table IX Frequency and Average Degree of Success 
for Minimum Time On-Orbit 


Level 

Frequency 

Average Degree of 
Success 

One day 

3 

8.7 

Two 

6 

5.8 

Four 

1 

5.0 

Five 

2 

8.0 

Six 

7 

8.5 

Seven 

79 

7.8 

Eight 

42 

8.2 

Nine 

53 

7.4 

Missing 

100 

7.6 


Table X Frequency and Average Degree of Success 
for Storage Location 


Level 

Frequency 

Average Degree of 
Success 

Locker 

73 

7.4 

Rack 

45 

7.2 

Pallet 

5 

8.4 

Self - Contained 

58 

8.1 

Other 

17 

7.6 

Not Applicable 

11 

8.3 

Missing 

84 

7.8 


Table XI Frequency and Average Degree of Success 
for Pad Weather 


Level 

Frequency 

Average Degree of 
Success 

11 °C - 15 °C 

11 

6.18 

16 °C - 20 °C 

9 

7.2 

21 °C - 25 °C 

142 

7.6 

26 °C - 30 °C 

129 

7.95 


1 

9 

Missing 

1 

8 
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Conclusions 
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Appendix A 

ANOVA for Each Factor 


Factor 

Levels 

Significance 

Level 

Number of 
Experiments Used 

Failure Detection 

4 

0.0001 

274 

Active 

2 

0.003 

293 

Altitude Requested 

10 

0.008 

188 

Experiment Type 

10 

0.01 

293 

Level of Ground Crew Involvement 

4 

0.018 

238 

Minimum Time On-Orbit 

8 

0.025 

193 

Storage Location 

6 

0.044 

209 

Level of Orbital Crew Involvement 

4 

0.091 

236 

Orbiter Pad Weather 

5 

0.094 

292 

Launch Delay Cause 

4 

0.232 

288 

Orbiter 

5 

0.255 

293 

Wait Time on Pad 

20 

0.261 

282 

Launch Delay Duration 

4 

0.312 

288 

Flight Altitude 

22 

0.362 

292 

1 Interface 

9 

0.407 

258 

1 Time Between Previous Flight 

29 

0.413 

292 

Flight Duration 

9 

0.440 

293 

No. of Experiments On-Board 

18 

0.470 

291 

No. of Lockers 

8 

0.597 

74 

Type of Orbital Crew Involvement 

29 

0.677 

228 

Experiment Location 

11 

0.833 

289 

Inclination Requested 

6 

0.868 

183 

Flight Inclination 

7 

0.880 

292 

Iteration 

9 

0.907 

285 

Organization 

6 

0.946 

289 

Crew Size 

6 

0.959 

293 
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Appendix B 
Factor Definitions 

1. Active (Yes/No): requires crew involvement or is automated 

2. Altitude Requested: altitude requested by principal investigator for optimal 

experiment performance 

3. Experiment Location: where the experiments are located during operation on the 

orbiter 

4. Experiment Type: type of experiment (ex. hardware /instruments, biological) 

5. Failure Detection: where problem was detected (on-orbit or post-flight) 

6. Inclination Requested: inclination requested by principal investigator for 

optimal performance 

'7. Interface: service provided by the orbiter which the experiment incorporates 

into its design 

8. Iteration: number of times the experiment has been executed on the orbit 

9. Level of Ground Crew Inv.: an estimate of the number of hours an orbital crew 

member works with an experiment divided by the experiment's total time of 
operation 

10. Level of Orbital Crew Inv.: an estimate of the number of hours a ground crew 

member can influence experiment operation divided by the experiment's total time 
of operation 

11. Minimum Time On-orbit: the time the principal investigator felt was needed to 

run an experiment 

12. Number of Lockers: number of lockers occupied by the experiment during operation 

13. Organization: group which developed experiment 

14. Storage Location: where the experiment was stored on-orbit prior to operation 

15. Type of Orbital Crew Involvement: lists specific activities required for 

experiment operation 

16. Crew Size: number of crew members for a particular mission 

17. Flight Altitude: altitude for a particular mission 

18. Flight Duration: duration of a particular mission 

19. Flight Inclination: inclination for a particular mission 

20. Launch Delay Cause: examples: weather, orbiter 

*21 

22. Number of Experiments On-board: number of experiments for a particular mission 

*23 

24. Orbiter Pad Weather: weather at time of launch 

25. Time Between Previous Flight: time between previous shuttle mission 

26. Wait Time on Pad: includes loading time and delays 

*21,23: Launch Delay Duration, Orbiter (self-explanatory) 
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